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(57) ABSTRACT 

What is disclosed is a method comprising performing an 
observation on a sensor having a plurality of pixels, for each 
of the pixels that are unclassified, determining a score 
according to the observation, if the score for the each pixel 
satisfies a stopping condition, classifying the each pixel as 
being one of either defective or functional, and repeating the 
steps of performing, determining and classifying for any the 
pixels remaining unclassified after determining the score. 

20 Claims, 5 Drawing Sheets 
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ROBUST SEQUENTIAL APPROACH IN pixels may be defective. The extra memory needed to store 

DETECTING DEFECTIVE PIXELS WITHIN toe defective pixel locations adds to the total cost/tiroe-to- 

AN IMAGE SENSOR manufacture of the device and also requires actual data 

transfer during the process of assembling/integrating the 
5 sensor into the device. The defective pixel locations must be 

BACKGROUND OF THE INVENTION separately stored prior to device assembly into a fixed 

1. Field of the Invention memory such as a hard disk. Once the defective locations are 
The invention relates generally to imaging. More stored signal processing techniques post image capture may 

specifically, the invention relates to detecting defective be used to correct the defective pixels. A more arbitrary way 

pixels in an image sensor 10 of corrcctlon ™age defects, which has also been utilized, is 

2. Description of the Related Art t0 n0t ^ defe 1 Clive F e \ but . ,real unknown <! efec ' s 

, . . t , as noise and apply an lmage-by-image noise removal tech- 

Imaging devices such as digital cameras and scanners nique to the entire sensor output (image) Whik avoiding 

may have as one component, an image sensor which is mem ory cost and data transfer during assembly such tech- 
manufactured as a CCD (Charge Coupled Device), CID 15 niques have the disadvantage of being computationally 
(Charge Injection Device) or CMOS (Complementary expensive to implement and of potentially reducing the 
Metal-Oxide Semiconductor) device. The image sensor is sha rpness of the image, which is a key to visual appearance, 
composed of an array of "sense" or pixel locations which r lL • , P . , , 
captures energy from a illuminant, often converting this , For lhese re , as f ons ' * 61 * 18 a . Qe ,<? f °' a meth ° d t0 d * ect 
energy into a concrete measure such as an intensity value. In 20 f " d «,mpensate for defective pixel locations without adding 

most cases, imaging sensors will have a certain number of 20 t0 t ? me/ f ost of ma ™ f acture ° f the dev ' ce and wllhout 

pixel locations which are "defective" due to fabrication or sacnficin 6 <P**Y or ad <>'ng to the computation 

manufacturing errors. It is extremely difficult, if not re «J™ 6nts du ™g ™W processing on the device m 

•ui 4 i j ■ t. r u • »■ i which the sensor is to be employed, 

impossible, to guarantee during such fabrication/ v J 

manufacturing that none of the pixels in the sensor will be n< cm » 4 A r, v 

, f A tt % c .. „ . . / . , , 25 SUMMARY 
detective, A defective pixel of a sensor is one which when 

exposed to an illuminant will produce a different intensity What is disclosed is a method comprising performing an 

value or response than that of a "fully functional" pixel when observation on a sensor having a plurality of pixels, for each 

exposed to that same illuminant. In other words, the defec- of the pixels that are unclassified, determining a score 

tive pixel is abnormally sensitive/unsensitive to light than a 30 according to the observation, if the score for the each pixel 

fully functional pixel. Such defects if not detected and then satisfies a stopping condition, classifying the each pixel as 

compensated for, may cause the captured image to be of less being one of either defective or functional, and repeating the 

visually perceived quality and if prominent can detract the steps of performing, determining and classifying for any the 

attention of the viewer towards the defective pixel(s). pixels remaining unclassified after determining the score. 

Defects in pixel locations can be split into three 35 

categories-Stuck High, Stuck Low and Abnormal Response. BRIEF DESCRIPTION OF THE DRAWINGS 

A Stuck High defective pixel is one which always responds ^ objccts> features and advantagcs of the mcthod and 

to the lighting condition by producing a high intensity value. apparatus for me preS ent invention will be apparent from the 

For instance, if the intensity of mxels ranges from 0 to a high following description in which: 

of 255, a Stuck High pixel would always respond to lighting 40 . n r . 

withavalueof255,evenifactualmeasuredintensit y forth a t J™: 1 15 a flow dla 8 ram of an embodiment of the 
location of the scene would be 25, for example, if captured 

by a functional pixel A Stuck Low defective pixel is one FIG - 2 shows the determining of an MND (Minimum 

which always responds to the lighting condition by produc- Neighboring Distance) according to an embodiment of the 

ing a low intensity value. A Stuck Low pixel may respond 45 invention. 

with a value of 5 even though a functional pixel would show FIG. 3 shows an exemplary probability distribution of 

the intensity value to be 200, 100 etc. A pixel with an MND values for one image. 

Abnormal Response defect has no absolute, but rather a FIG. 4 is a representation of the number of observations 

relative variance from a functional pixel. For instance such (i. e ,, scanned images) needed for an accurate determination 

a pixel would inaccurately respond by a particular 50 of classification of a pixel. 

percentage such that for instance, were a functional pixel FIG. 5(a) illustrates a weighting function for a Stuck Low 

would read a value X, the Abnormal Response defective defect 

pixel would respond with a value 0.25 *X. The Abnormal „ " .„ , A r, t TT . , 

Response is thus proportional or relative to the intensity defec? imj strates a weighting function for a Stuck High 

being captured, rather than an absolute high or low. Pixels 55 ec ' 

exhibiting any of these types of defects should, desirably, be FIG - 6 is a block diagram of an image processing appa- 

corrected or compensated for. ratus according to an embodiment of the invention. 

The first step in any such compensation is the detection of FIG - 7 is a system diagram of one embodiment of the 

which pixels are in fact "defective". Conventionally, such invention, 

detection is performed by identifying the defective pixel 60 
locations in a controlled environment, such as during quality 
control for the sensor as a whole, after the sensor is 

fabricated. The identified locations are recorded and then Referring to the figures, exemplary embodiments of the 

transferred to some non-volatile memory on the device in invention will now be described. The exemplary embodi- 

which the sensor is used such as on a digital camera. In 65 ments are provided to illustrate aspects of the invention and 

modern "mega-pixel" image sensors, where the total size of should not be construed as limiting the scope of the inven- 

the sensors have on the order of 1000 by 1000 pixels, many tion. The exemplary embodiments are primarily described 
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with reference to block diagrams or flowcharts. As to the tion need be stored (step 170) with the default that all other 

flowcharts, each block within the flowcharts represents both locations are functional. 

a method step and an apparatus element for performing the Once all unclassified pixels in given observation (scanned 

method step. Depending upon the implementation, the cor- image) are processed (checked at step 135) then the proce- 

responding apparatus element may be configured in s dure returns to step 100. Again, at step 100, if unclassified 

hardware, software, firmware or combinations thereof. pixels still remain even after processing of pixels using the 

FIG. 1 is a flow diagram of an embodiment of the last scanned image, the next image is canned in (step 110) 

invention. and steps 120-170 are repeated. Each scanned image rep- 

In statistical analysis, an approach known as the Sequen- resents a statistically relevant observation, and on each 

tial Probability Ratio Test (SPRT), set forth in "Sequential io observation, only those pixels whose scores did not satisfy 

Analysis", A. Wald (Wiley & Sons, New York 1947), has a stopping condition are reprocessed. This continues until all 

been developed. In the SPRT approach, an inference is made < or "J 08 * dc P endm g on desired design) pixel locations are 

which attempts to identify one of two hypotheses from a classified. 

sequence of observations. For instance, in the defective pixel FIG * 2 shows the determining of an MND (Minimum 

detection case, the hypotheses may be the presence of a 15 N^oring Distance) according to an embodiment of the 

defective pixel and the presence of a functional pixel. When invention. 

an adequate number of observations are made, a decision A minimum neighboring distance (MND) for a given 

may be arrived at as to which of the two hypotheses fit. This P* cl Xs F^l be deter ™ e u d b y considering the "distance; or 

decision may or may not be accurate since it is based upon *f ^ b «™ n ™* hb ™ n Z P™1 values and that given 

a statistical approximation. THe goal then in such approaches 20 1 v v ' v °T ™ hbo *°° d °. f e f ht **}™* P^els 

j Xi * * r f i a * *• *l j ■• a,, Xo, X 4 , X 6 , X 7 , X s and Xo about a pixel X,. For a 

is to reduce the impact of false detections upon the decision mono ^ hr 7 m ^ or 7 des f red ^ the £ the 

space. Such approaches also work best when the content of absolute valuc diffcrcnce) betwce £ cach of those ne \ gh b ors 
the observed samples is known a prion. The software and the central pixel X 5 is computed. The minimum of these 
approach to defective pixel detection, in one embodiment, distances becomes the MND for that pixel X s . Alternatively, 
may impose on the end user the task of initiating the 2 5 a neighborhood consisting of only north, south, east and 
detection/correction process. Since the content of images to west neighbors, X2, X8, X6 and X4, respectively, may be 
be scanned in during this detection process is unknown, the considered in determining the MND. In a Bayer pattern 
SPRT must be made robust such that it dynamically changes sensor, it may be desirable to consider only those neighbor- 
according to measured observations. ing pixel, in the same color plane (R, G or B). In a sensor 
With this framework in mind, FIG. 1 illustrates a logic 30 array that has a captured intensity value at each location, it 
flow that can achieve defective pixel detection. Initially it is is likely that closely neighboring pixels should have the 
assumed that all the pixel locations are unknown or inde- same or similar intensity values with those of their neigh- 
terminant (i.e., classified neither defective nor functional). bors. This assumes that the neighboring pixels contain no 
Accordingly, the condition at step 100 would be satisfied. defects (see FIG. 3 and associated description for description 
When some pixels remain unclassified, the first step in SPRT 35 of defects). This is especially true with large sensors or a 
classification is to scan in an image using a device that is small field of view during image capture or both. The MND, 
already constructed with and incorporates the sensor array also denoted as Y, tells roughly how close to neighboring 
under consideration (step 110). Each scanned image would pixels a certain pixels response is. For example, Y may be 
represent a new observation upon which measurements may given as Y«min(|X 5 -Xj) 1^(1,2,3,4,6,7,8,9} for a mono- 
be made. After an image is captured into the sensor array, the 40 chrom sensor, or Y=min(|X 5 -X Jfc 0 ^{1,3,7,9} for a Bayer 
image is scanned for information content which can be pattern sensor with X 5 as a Green pixel. For instance, 
measured (step 120) such as a readout of the intensity value consider the following intensity values X^lOO, X^O, 
for each pixel. According to one embodiment of the X 3 =95, X 4 =105, X^-llO, X 7 «85, X 8 =80 and X^S. If X 5 , 
invention, a measurement is performed by considering the the pixel under consideration, has an intensity value of 102, 
minimum neighboring distance (MND) of surrounding pix- 45 the minimum neighboring distance would be (102~100)-2, 
els (see FIG. 2) whether the neighborhood is composed of all and in reference to FIG. 3, the value of "b", the intensity 
monochrome pixels or only those in a particular color plane value of the pixel location (XJ yielding the MND would be 
of a Bayer Pattern CFA (Color Filter Array). Next, the image 100. 

statistics, primarily, the mean, variance and/or standard FIG. 3 shows an exemplary probability distribution of 

deviation, of the values are measured (step 130) from the 50 MND values for one image. Functional pixels will most 

MND. For all unclassified pixel locations (checked at step likely, in a statistical sense, not deviate far from a neigh- 

135), a "score" is then determined (step 140) location by boring pixel and will probably have the same response, 

location. The score is based upon the estimated likelihood of Thus, where the MND is zero, for instance, the pixel under 

a pixel being defective. If this score satisfies a particular consideration can with a high (perhaps the highest) degree of 

stopping condition (checked at step 150), then the pixel 55 certainty be classified as functional. Thus, the distribution of 

under consideration may be classified accordingly as either Y, which is plotted in FIG. 3 is equal to P 0 (y) when the 

being defective or functional (step 160) and further may functionality of the pixel is most likely to be guaranteed. The 

have a designation as to the type of defect. If not, then distribution of Y, the MNDs, for functional pixels has been 

another unclassified pixel is processed (repeat from step found on experimentation to resemble the well-known 

135). Advantageously, when a pixel is capable of being 60 Gamma distribution. Based on image statistics, likewise, a 

classified, the defective pixel locations may all be stored into gamma distribution curve of the defective pixel classifica- 

host computer or external device (step 170) so that correc- tion hypothesis may be derived. The center and exact 

tive measures such as noise reduction may be undertaken on parameters of the gamma distribution will vary depending 

only those pixel locations judged to be defective. This also upon which type of defect is under concern. If "b" is the 

eliminates the need for storing defective pixel location data 65 intensity value of that neighboring pixel which yielded the 

on the sensor device, camera and also the need for deter- MND, then the center C of a distribution for a defect can be 

mining it during manufacture. Further, only defective loca- modeled by: 
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Stuck High - - - C ~~ \nS5-b\ 
Stuck Low - - - C — \b\, and 
Abnormal Response - - - C — 0.15*fc. 

The abnormal response defect is assigned an exemplary 
15% deviation but may be any value as desired or as 
motivated by sensor properties or design constraints. In a 
highly spatially correlated image, a functional pixel will 
have its MND distribution centered about zero or close to 
zero and will rapidly diminish with increasing values. The 
values given above assume an eight bit (ranging from 0 to 
255) intensity value at each pixel location. For a stuck high 
defect, the value C is very close to 255, while for a stuck low, 
the center of the distribution is approximately b according to 
the definition of the MND. An abnormal response is a pixel 
that while not stuck high or stuck low, may be out of an 
acceptable tolerance. Pjfr) is the probability of the hypoth- 
esis of defect being true while P 0 (y) is the probability of the 
other hypothesis that of a functional pixel being true. If the 
distributions as shown in FIG. 3 overlap, any MNDs therein 
are very uncertain and should contribute little, if at all, to the 
overall score. The generalized gamma distribution, which 
the MNDs of functional pixels approximately follow, may 
be represented as: 

PM= cJ y -f$ r ****y-o.i...L 

where 0<a^l, y is the MND, T(ct) is the Euler's gamma 
function, L the maximum possible MND, and "c", is a 
scaling factor. The a and X are computed as a^n^/a 2 and 
X^m/a 2 , where m is the mean of the measured MNDS, and 
a 2 the variance. 

FIG. 4 is a representation of the number of observations 
(i.e., scanned images) needed for an accurate determination 
of classification of a pixel. According to SPRT theory, there 
is in a binary hypothesis scheme, a likelihood ratio that can 
be measured. The likelihood ratio is P!(y)/P 0 (y) or the 
probability of the observed pixel having condition 1 
(defective) divided by the probability of the observed pixel 
having condition 0 (functional). This likelihood ratio is a 
representation of the level of uncertainty regarding the 
measurements (MNDS) performed for a given pixel. Where 
the results are uncertain, more data (observations) in the way 
of more scanned in images may be needed. At each image 
scan and analysis, the likelihood ratio is multiplied together 
with past likelihood ratios, since each observation is an 
independent probabilistic event. This " accumulated" likeli- 
hood ratio is the "score" used to determine whether or not a 
pixel can be properly classified as defective or functional. If 
the logarithm of the accumulated likelihood ratio is 
considered, then the individual likelihood ratio at each 
observation may be added with previous likelihood ratios to 
determine and keep track of the score. 

FIG. 4 shows a graph of the log of the likelihood ratio 
versus the number of desired observations. Two (or more) 
boundary or stopping conditions may be defined for the 
sensor array, a condition A wherein the pixel can be likely 
accurately classified as functional and a condition B wherein 
the pixel upon reaching that boundary, the pixel may likely 
be classified accurately as defective. Assuming a sequence 
of observations yl, y2 . . . yn are needed for a given pixel, 
the log of the likelihood ratio (i.e., log (Pi(y)/Po(y))) niay 
increase or decrease until it satisfies one of the two stopping 
conditions A and B. When no images (n-0) have yet been 



15,318 Bl 

6 

captured, the likelihood ratio is one, therefore the log is zero 
since it is equally probable that the given pixel is defective 
as it is functional without any prior observations or knowl- 
edge regarding the sensor. As each observation is made, the 

5 uncertainty diminishes. The likelihood ratio is greater than 
one when the pixel under consideration is more likely to be 
defective (i.e., Pi(y)>P 0 (y)), and less than one when the 
pixel under consideration is more likely to be functional 
(i.e., Pi(y)<P 0 (y))- The number of observations required for 

io each pixel location depends upon the reliability or charac- 
teristics of neighboring pixels and thus may vary from image 
region to image region. In one region, it may be possible to 
reach a stopping condition with very few observations and 
in another, with more observations. The number of obser- 

15 vations needed depends upon the image content and the type 
of defective pixel desired to be detected. Not all images that 
observed are equal in response. Where non-functional (i.e. 
defective) pixels are concerned, the center or peak location 
of the distribution of the MNDs may be more dependent 

20 upon the local neighboring pixel intensities. If a user is 
performing the pixel detection, the user may be capturing 
images of different scenes or the same scene with different 
lighting which makes the uncertainty problem worse since 
each subsequent observation in terms of the neighboring 

25 intensities of a given pixel may be different in nature, To 
compensate for this problem and add robustness to the 
SPRT, a weighting procedure may be incorporated as 
described below. 

FIG. 5(a) illustrates a weighting function for a Stuck Low 

30 defect. 

Since the content of each image (observation) is 
unknown, there may be conditions which yield abnormal 
likelihood ratios at those observations. For instance, if for a 
given observation, the P 0 (y) value is extremely low and 

35 approximately equal to zero, then that observation's likeli- 
hood ratio would approach infinity. As such, if this mea- 
surement is added to the accumulated likelihood ratio 
(score), then the score would abruptly increase, perhaps 
yielding a false conclusion. For instance, consider that the 

40 center of the distribution for a stuck low defect is approxi- 
mately "b", the intensity value of the pixel yielding the 
MND, and thus the determination of the pixel score is 
dependent upon the neighboring pixel intensity. If a stuck 
low pixel were surrounded by low neighboring pixel inten- 

45 sities then it would be more difficult to decide based upon 
that observation whether that pixel is stuck low or not. 
Neighboring pixel intensity may be defined be "b", the 
intensity value of the pixel location yielding the MND. 
Where neighboring pixel intensity is in the high end for 

50 yet another observation of that same stuck low pixel 
location, the likelihood ratio will be more meaningful and 
thus can be afforded more emphasis. To offset abnormal 
observations, each likelihood ratio obtained at each obser- 
vation is weighted rather than being included in equal 

55 measure. The weighting function W <Si (y) which varies from 
a minimum of zero (excluding the observation in the score) 
to a maximum of one (including the observation with full 
emphasis). As shown in FIG. 5(a), above a certain threshold 
neighboring intensity, individual likelihood ratios are given 

60 full or almost full emphasis and thus have weighting equal- 
ing or approaching one. Below that threshold, when the 
neighboring intensity is low, the emphasis on the observa- 
tion is diminished and may even be weighted by a zero 
which would exclude the observation from the accumulated 

65 likelihood ratio. 

In FIG. 5(6), a weighting function for a Stuck High defect 
is illustrated. A Stuck High pixel is one who returns a high 
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intensity value for a given illurninant, near the very top of resulting tally of row and column indices of the defective 
the range of possible intensities, even though the same pixel locations may be loaded into RAM 616 via the bus 660 
illuminant when exposed to a functioning pixel, would which facilitates data transfer in bi-directional capacity 
return a low or medium intensity value, When detecting between the imaging apparatus and external devices. In an 
Stuck High defect, an observed likelihood is more relevant 5 alternate embodiment substitution unit 615 may be elimi- 
where the neighboring intensity is not high. Thus, in a nated from the imaging apparatus in favor of some form of 
weighting function for Stuck High defects, an observation post-processing, such as filtering or averaging, after the 
where the neighboring intensity is high is given, according image is downloaded or is ready to be displayed, 
to a embodiment of the invention, less emphasis or maybe In the imaging apparatus, companding module 625 is 
excluded from the accumulated likelihood ratio (i.e., the 10 designed to convert each original pixel of n-bit (typically 
score) altogether by being given zero emphasis. Below a n-10) intensity captured from the sensor to an m-bit inten- 
certain threshold, the weighting factor increases to one or sity value, where m<n (typically, m=8). Companding mod- 
near one consistent with the desired effect that observations ule 625 is not needed if the sensor 600 and capture interface 
where neighboring intensity is low should be given more or 610 provide a standard 8-bit per-pixel value. Defective pixel 
full emphasis in the score. 15 detection should be performed in accordance with the rel- 
In detecting defective pixels, a first hypothesis will claim evant intensity range of the sensor which if originally is a 
that a particular pixel location is functional or non-defective higher bi-resolution such as 10-bit, should consider the high 
while a second will claim that the location is defective. In end of the intensity range to be 1023 and not 255 (for 8 -bit 
such probabilistic determinations, there is a likelihood that intensity values) as discussed above, 
the decision made or hypothesis proven is incorrect even 20 A primary compressor 628 receives companded sensor 
though the conditions for determination were satisfied. image data and performs image compression such as JPEG 
Incorrect results take the form of either a "false detection" or DWT. ARAM 629 can be used to store DWT coefficients 
or "miss detection." A "false detection" would result in a and/or quantization thresholds for each channel/sub-band as 
functional pixel being classified as defective while a "miss desired in executing such compression. Primary compressor 
detection" would result in a defective pixel being classified 25 628 can be designed to provide outputs which are sensitive 
as functional. In mapping out an image sensor for defective or corrective of defective pixel locations and their values, 
pixels, as long as defective pixels are detected properly, it is sending such compressed values to Encoder/Data Packer 
acceptable to have a small number of functional pixels that 630. 

are mis-classified as being defective Thus, in detecting, the Each of the RAM tables 616, 626, 629 and 631 can 

number of false detections are to be minimized in relation to 30 directly communicate with a bus 660 so that their data can 

miss detections which are to be avoided altogether. be loaded and then later, if desired, modified. Further, those 

FIG. 6 is a block diagram of an image processing appa- RAM tables and other RAM tables may be used to store 

ratus according to an embodiment of the invention. intermediate result data as needed. When the data in storage 

FIG. 6 is a block diagram of internal image processing arrays 640 is ready to be transferred external to the imaging 

components of an imaging device incorporating at least one 35 apparatus of FIG. 6 it may be placed upon bus 660 for 

embodiment of the invention including an adaptive encoder. transfer. Bus 660 also facilitates the update of RAM tables 

In the exemplary circuit of FIG. 6, a sensor 600 generates 616, 626, 629 and 631 as desired. Depending on the design 

pixel components which are color/intensity values from of the apparatus, a diagnostic capture mode may be provided 

some scene/environment. The n-bit pixel values generated that performs limited or no compression and no data cor- 

by sensor 600 are sent to a capture interface 610. Sensor 60 0 40 rection while the identifying defective pixels is being under- 

in the context relating to the invention will typically sense taken. 

one of either R, G, or B components from one "sense" of an FIG. 7 is a system diagram of one embodiment of the 

area or location. Thus, the intensity value of each pixel is invention. 

associated with only one of three (or four if Gl and G2 are FIG. 7 illustrates a computer system 710, which may be 

considered separately) color planes and may form together 45 any general or special purpose computing or data processing 

a Bayer pattern raw image. These R, G and B color "chan- machine such as a PC (personal computer), coupled to a 

nels" may be compressed and encoded separately or in camera 730. Camera 730 may be a digital camera, digital 

combination, whichever is desired by the application. Cap- video camera, or any image capture device or imaging 

ture interface 610 resolves the image generated by the sensor system, or combination thereof and is utilized to capture an 

and assigns intensity values to the individual pixels. The set 50 image of a scene 740. Essentially, captured images are 

of all such pixels for the entire image is in a Bayer pattern processed by an image processing circuit 732 so that they 

in accordance with typical industry implementation of digi- can be efficiently stored in an image memory unit 734, which 

tal camera sensors. may be a RAM or other storage device such as a fixed disk. 

It is typical in any sensor device that some of the pixel The image(s) contained within image memory unit 734 that 

cells in the sensor plane may not respond to the lighting 55 is destined for computer system 710 can be according to one 

condition in the scene/environment properly. As a result, the embodiment of the invention, for the purpose of determining 

pixel values generated from these cell may be defective. the defective pixel locations of the camera 730. In most 

These pixel locations are called "defective pixels." In one digital cameras that can perform still imaging, images are 

embodiment of the invention, a "pixel substitution" unit 615 stored first and downloaded later. This allows the camera 

may replace the value read out in each dead pixel by the 60 730 to capture the next object/scene quickly without addi- 

intensity value immediate previously valid pixel in the row. tional delay. The use of a computer system 710, particularly 

A RAM 616 consists of the row and column indices of the in detecting defective pixel locations of the camera, reduces 

dead pixels, which are supplied by the defective pixel the computation/storage requirements of the camera 730 

substitution methodologies presented in various other allowing for a less complex and thus, more inexpensive 

embodiments. The methodology for detecting which pixels 65 manufacture. 

in the sensor are defective may be carried out by a computer The image processing circuit 732 carries out the pixel 

system or other such device as illustrated in FIG. 7. The substitution (if any), companding and compression of 
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images captured by the camera 730. When a captured image, What is claimed is: 

whether compressed or raw, is downloaded to computer 1. A method of detecting defective sensors in a sensor 

system 710, it may be decoded and then rendered to some arrav comprising: 

output device such as a printer (not shown) or to a monitor performing an observation of an object on a sensor array 

device 720. The execution of the defective pixel detection * havin S a plurality of pixels, each pixel corresponding to 

methodology described above, and image decompression, if a sens ° r ^ sai <! sen t sor arra y and each sensor generating 

needed, may be achieved using a processor 712 such as the a valuc for ^ corresponding pixel; 

Pentium™ processor with MMX Technology (a product of for each of said pixels, determining a score based on 

Intel Corporation) and a memory 711, such as RAM, which statistical analysis of said pixel values using said obser- 

is used to store/load instruction addresses and result data. 10 vation; and 

The applications) used to perform the detecting of defec- if said L score f° T r said each P^ 1 satisfies a stopping 

tive pixels based on a series of images downloaded from condition classifying said each pixel as being one of 

camera 730 may be from an executable compiled from „ either defective or functional. 

source code written in a language such as C++. The instruc- 2 ' A accordin g to claim 1 wherein determining a 

tions of that executable file, which correspond with instruc- 15 score inciudes: 

tions necessary to scale the image, may be stored to a disk measuring image content; and 

718 or memory 711. Further, such application software may compiling statistics of said image content. 

be distributed on a network or a computer-readable medium 3. A method according to claim 2 wherein said image 

for use with other systems. content includes pixel values. 

When an image, such as an image of a scene 740, is 20 . 4. A method according to claim 3 wherein said statistics 

captured by camera 730, it is sent to the image processing mclude one or more of mean ' var iance and standard devia- 

circuit 732. Image processing circuit 732 consists of ICs and Uon measurements as a function of image content measured 

other components which may execute, among other **y ™ P K ™ values. 

functions, the compression of the image pixel data set 25 5. A met h°°- according to claim 1 wherein determining a 

compensated by defective pixel correction. The image sc °re includes. 

memory unit 734 will store the compressed/encoded image computing an individual likelihood ratio for each said 

data. Once all pixels are processed and stored or transferred observation; and 

to the computer system 710 for rendering the camera 730 is accumulating said individual likelihood ratios into said 

free to capture the next image. When the user or application 3Q score. 

desires/requests a download of images, the encoded image 6. A method according to claim 5 wherein said individual 

data in the image memory unit, are transferred from image likelihood ratio is formed by a ratio of the probability that 

memory unit 734 to the I/O port 717. I/O port 717 uses the said eacn P ixel is functional to the probability that said each 

bus-bridge hierarchy shown (I/O bus 715 to bridge 714 to pixel is defective. 

system bus 713) to temporarily store the data into memory 35 7. A method according to claim 6 wherein said individual 

711 or, optionally, disk 718, Computer system 710 has a likelihood ratios are weighted in accordance with the rel- 

system bus 713 which facilitates information transfer evance of said observation. 

to/from the processor 712 and memory 711 and a bridge 714 8 * A method according to claim 7 wherein said relevance 

which couples to an I/O bus 715, I/O bus 715 connects is a function of neighboring pixel intensity, 

various I/O devices such as a display adapter 716, disk 718 4Q 9 - A method according to claim 6 wherein said probabili- 

and an I/O port 717, such as a serial port. Many such 40 ^ es are determined by measuring at minimum neighboring 

combinations of I/O devices, buses and bridges can be distance from said each pixel. 

utilized with the invention and the combination shown is 10. A method according to claim 9 wherein said minimum 

merely illustrative of one such possible combination. neighboring distance is determined by: 

According to one embodiment of the invention, the 45 by computing absolute value of the pixel value differences 

detecting of defective pixels may be achieved on computer between said each pixel and each of the neighboring 

system 710 by downloading and analyzing a series of pixels; and 

images from camera 730. The defective pixel location data finding a minimum among said absolute value of pixel 

may be stored in a disk, memory 711 or other storage valuc differences. 

mechanism and can be used in perform correction or 50 11. An article comprising a machine-readable medium 

enhancement of any downloaded image. Post download having instructions stored thereon which when executed 

correction of defective image locations eliminates the need causes: 

for incorporating such features into the camera 730. Com- determining a score for each of a plurality of pixels, said 

puter system 710 during pixel detection will perform such pixels corresponding to an observation of an obect by 

computations as the computing of likelihood ratios, weight- 55 sensors of a sensor array said score being based on 

ing and accumulation of same as well as comparison of statistical analysis of pixel values generated by said 

scores with boundary or stop conditions, and computation of sensors according to said observation; and 

MNDs. The computer system is also more ideally suited to if said score for said each pixel satisfies a stopping 

storing the large number of data points required in compiling condition, classifying said each pixel as being one of 

a histogram or distribution of values. 60 either defective or functional. 

The exemplary embodiments described herein are pro- 12. A system comprising: 

vided merely to illustrate the principles of the invention and a memory capable of storing instructions and data related 

should not be construed as limiting the scope of the inven- to defective pixel detection; 

tion. Rather, the principles of the invention may be applied a processor coupled to said memory, said processor 

to a wide range of systems to achieve the advantages 65 adapted to execute said instructions and manipulate 

described herein and to achieve other advantages or to said data to determine which pixels of an image sensor 

satisfy other objectives as well. are defective, said instructions including: 
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for each of said pixels that are unclassified, determining 
a score based on statistical analysis of pixel values 
generated by said image sensor according to an 
observation; and 

if said score for said each pixel satisfies a stopping 
condition, classifying said each pixel as being one of 
either defective or functional. 

13. A system according to claim 12 further comprising: 
an image sensor array providing data in the form of pixel 

values, each pixel corresponding to a sensor of said 
sensor array, said sensor array adapted to scan an image 
for each desired observation. 

14. A system according to claim 12, comprising: 

a defective pixel location memory adapted to store loca- 
tions of pixels determined defective. 

15. A system according to claim 13, wherein said image 
sensor scans are adjusted according to said pixels classified 
as defective. 

16. A system according to claim 12, wherein determining 
a score comprises, compiling statistics of pixel values, 
wherein said statistics include one or more of mean, variance 
and standard deviation measurements as a function of image 
content measured by the pixel values. 

17. A system according to claim 12, wherein determining 
a score includes: 

computing an individual likelihood ratio for each said 

pixel for said observation; 
accumulating said individual likelihood ratios into said 

score wherein said individual likelihood ratio is formed 
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by a ratio of the probability that said each pixel is 
functional to the probability that said each pixel is 
defective wherein said individual likelihood ratios are 
weighted in accordance with neighboring pixel values. 

18. A system according to claim 17, wherein said prob- 
abilities are determined by measuring at minimum neigh- 
boring distance from said each pixel wherein said minimum 
neighboring distance is determined by computing absolute 
values of the pixel value differences between said each pixel 
and each of said neighboring pixels and finding a minimum 
among said absolute values of pixel value differences. 

19. An article according to claim 11, wherein determining 
a score comprises, compiling statistics of pixel values, 
wherein said statistics include one or more of mean, variance 
and standard deviation measurements as a function of image 
content measured by the pixel values. 

20. An article according to claim 19, wherein determining 
a score includes: 

computing an individual likelihood ratio for each said 
pixel for said observation; 

accumulating said individual likelihood ratios into said 
score wherein said individual likelihood ratio is formed 
by a ratio of the probability that said each pixel is 
functional to the probability that said each pixel is 
defective wherein said individual likelihood ratios are 
weighted in accordance with neighboring pixel values. 



06/09/2004, EAST Version: 1.4.1 



UNITED STATES PATENT AND TRADEMARK OFFICE 



CERTIFICATE OF CORRECTION 



PATENT NO. 
DATED 



: 6,625,318 Bl 

: September 23, 2003 



Page 1 of 1 



INVENTOR(S) :Tanetal. 

It is certified that error appears in the above- identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Column 5 , 

Line 1, delete "|1255-b|", insert |255-b| --. 
Column 10, 

Line 38, delete "intensity", insert ~ values --. 
Line 55, after "sensor array", insert - , 



Signed and Sealed this 



Seventeenth Day of February, 2004 




JON W. DUD AS 
Acting Director of the United States Patent and Trademark Office 



06/09/2004, EAST Version: 1.4.1 



